A Self-Organizing Storage Cluster for Decentralized and Reliable File Storage*

This paper describes the design and implementation of a decentralized storage cluster for file storage so as to provide self-organizing, available, scalable, and consistent data management capability for cluster platform. The self-organizing capability of CHT grants our storage cluster the potential to handle both failure and online provisioning gracefully. We use soft-state to manage membership changes, and evenly distribute data across nodes by adopting linear hash algorithm so as to achieve incremental scalability of throughput and storage capacity, thus making the system easy to scale as the number of nodes grows. In order to guarantee metadata and data strict consistency among multiple replicas, we adopt decentralized weighted voting scheme that allows data and metadata operations to be safely interleaved, with enabling the system to perform self-tuning. We also present the experiment results to demonstrate the features and performance of our design. * This research is supported by National 973 Great Research Project of P.R China under the grant No. 2004CB318200 and National Natural Science Foundation under grant No. 60273037 and No. 60303031. 78 A Self-Organizing Storage Cluster for Decentralized GESTS-Oct.2005